Camera flash mitigation

ABSTRACT

Techniques are generally described for an image capture system that may include an image sensor, a flash for providing illumination, a data storage, and a processor operatively associated with the data storage. The processor may be adapted to execute computer implemented instructions to pre-store one or more image capture device characteristics in the data storage, acquire data in a pre-capture phase, model shadow effects based on either or both of the pre-stored data and the acquired data, modify one or more image capture device settings based on the modeled shadow effects, and record image data with the image sensor. Illumination may be provided substantially coincident with recording of the image data.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of pending U.S. patent applicationSer. No. 12/489,197, filed on Jun. 22, 2009. This application isincorporated herein by reference in their entirety and for any purpose.

BACKGROUND

Photographic systems produce a wide range of image quality when operatedby amateur photographers. If the photographic environment for a givenscene is well suited to the photographic system (e.g. ambient lightlevel is uniform and of sufficient intensity), good results aretypically obtained. However, when these conditions are not present,image defects may be introduced due to failures in the capture orreproduction system, thereby reducing the quality of the final viewedimage. To minimize the effects of suboptimal image capture conditions,photographic systems have attempted to compensate by adding featuresintended to expand the range of light levels and distances where imagesmay be captured. For example, if the intensity of the ambient light isinsufficient to provide adequate exposure, and the primary subject islocated less than a predetermined distance from the camera, mostbuilt-in electronic flash units are able to provide auxiliaryillumination sufficient to at least partially expose the primarysubject. However, even if the primary subject now receives adequateillumination, the flash may introduce image defects.

The image defect of shadowing may occur when auxiliary illumination isemployed. Specifically, light from an auxiliary illumination source, orflash, may not reach one or more background objects due to obstructionby the photographic subject, and may appear in the final image as adarkened area behind the photographic subject, which is oftenundesirable in terms of image quality. This objectionable phenomenon isunderstood to be visible in the final image at least in part because ofa displacement between the lens and the flash mounted in, or to, thecamera housing.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other features of the present disclosure will becomemore fully apparent from the following description and appended claims,taken in conjunction with the accompanying drawings. Understanding thatthese drawings depict only several examples in accordance with thedisclosure and are, therefore, not to be considered limiting of itsscope, the disclosure will be described with additional specificity anddetail through use of the accompanying drawings, in which:

FIG. 1 depicts a block diagram of an image capture system such as adigital camera apparatus in accordance with some examples.

FIG. 2 depicts a computing system for implementing examples of thepresent disclosure.

FIG. 3 depicts a process flow diagram of operation of a flash mitigationsystem in accordance with some examples.

FIG. 4 a depicts a camera suitable for use with systems in accordancewith some examples.

FIG. 4 b depicts a diagram of an image capture scene in accordance withan illustrative example.

FIG. 5 depicts a block diagram of an example computer program product inaccordance with the present disclosure.

FIG. 6 a depicts a computer system including a processor configured forperforming an example of a method for minimizing shadow effects within adigital image in accordance with some examples.

FIG. 6 b depicts a computer system including a processor configured forperforming an example of a method for minimizing shadow effects within adigital image in accordance with some examples.

FIG. 7 depicts a schematic diagram of an image capture system inaccordance with some examples.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof. In the drawings,similar symbols typically identify similar components, unless contextdictates otherwise. The illustrative examples described in the detaileddescription, drawings, and claims are not meant to be limiting. Otherexamples may be utilized, and other changes may be made, withoutdeparting from the spirit or scope of the subject matter presentedherein. It will be readily understood that the aspects of the presentdisclosure, as generally described herein, and illustrated in theFigures, may be arranged, substituted, combined, separated, and designedin a wide variety of different configurations, all of which areexplicitly contemplated herein.

In some examples, the present disclosure may relate to pre-processingsystems and methods of mitigating defects in captured digital image dataresulting from the use of an auxiliary illumination source. Moreparticularly, in an illustrative example, the present disclosure mayrelate to mitigation of shadow effects in captured digital image dataresulting from the use of an auxiliary illumination source. Suchprocesses and methods may, for example, enable amateur photographers totake higher quality photos without requiring time consuming andexpensive post-processing techniques. In further examples, data obtainedduring pre-processing may be used in post-processing.

FIG. 1 depicts a block diagram of an image capture system such as adigital camera apparatus in accordance with some examples of the presentdisclosure. The image capture system, hereinafter referred to as camera10, may include a processor 30. It is appreciated that many of theprocesses implemented in the camera 10 may be implemented in orcontrolled by software operating in a microprocessor, central processingunit (CPU), controller, digital signal processor (DSP) and/or anapplication specific integrated circuit (ASIC), collectively depicted asblock 30 and termed as “processor”. In various examples, more or fewercomponents may be included in processor 30. Peripheral components, suchas components for direct user viewing or manipulation, and including,for example, such as buttons and a display may be included. Generally, auser interface (UI) 40 and/or the peripheral components 45 may becontrolled by a micro-controller 32. The processor 30, in response to auser input at micro-controller 32, such as for example pressing ashutter button or other suitable means, may initiate and control thedigital photographic process.

With reference to FIG. 2, depicted is a computing system forimplementing examples of the present disclosure. The computing system ofFIG. 2 includes a computer 101, including a central processing unit(CPU), also referred to as a processor, 102, main memory 103 and one ormore bulk storage devices 104. The processor 102 may generally be of anydesired configuration including but not limited to a microprocessor(μP), a microcontroller (μC), a digital signal processor (DSP), or anycombination thereof. Thus, individual processors 102 may include logicfor executing program instructions as well as other functional blockssuch as an arithmetic logic unit (ALU), a floating point unit (FPU), adigital signal processing (DSP) core, registers, accumulators, etc. Themain memory 103, which may be any suitable form of memory including, butnot limited to, volatile memory such as random access memory (RAM),non-volatile memory such as read only memory (ROM) and flash memorystorage, data storage devices such as magnetic disk storage (e.g., harddisk drive or HDD), tape storage, optical storage (e.g., compact disk orCD, digital versatile disk or DVD), or other machine-readable storagemediums that may be removable, non-removable, volatile or non-volatile.An algorithm for reducing shadow effects within a digital image may beprovided in the main memory 103, such as, for example, in the ROM.

The bulk storage devices 104 and their associated computer storage mediaprovide storage of computer readable instructions, data structures,program modules and other data for the computer 101. The bulk storagedevices 104 may also include an operating system 106, applicationprograms 107, program modules 108, and a database 180. The computer 101further includes user input devices 190 through which a user may entercommands and data. Input devices may include an electronic digitizer, amicrophone, a keyboard and pointing device, commonly referred to as amouse, trackball or touch pad. Other input devices may include ajoystick, game pad, satellite dish, scanner, or the like.

These and other input devices may be coupled to the processor 102through a user input interface that is coupled to a system bus, but maybe coupled by other interface and bus structures, such as a parallelport, game port or a universal serial bus (USB). Computers such ascomputer 101 may also include other peripheral output devices such asspeakers, which may be coupled through an output peripheral interface194 or the like.

The computer 101 may operate in a networked environment using logicalconnections to one or more computers, such as a remote computer coupledto network interface 196. The remote computer may be a personalcomputer, a server, a router, a network PC, a peer device or othercommon network node, and may include many or all of the elementsdescribed above relative to computer 101. The remote computer may beconsidered the other of the client or the server depending on thedesignation of the computer 101. Networking environments are commonplacein offices, enterprise-wide area networks (WAN), local area networks(LAN), intranets and the Internet. Source and destination machines neednot be coupled by a network 109 or any other means, but instead, datamay be migrated via any media capable of being written by the sourceplatform and read by the destination platform or platforms. In someexamples, the network may be a wireless network such as a wireless localarea network (WLAN). When used in a LAN or WLAN networking environment,the computer 101 is coupled to the LAN through a network interface 196or an adapter. When used in a WAN networking environment, computer 101typically includes a modem or other means for establishingcommunications over the WAN, such as the Internet or network 109. Itwill be appreciated that other means of establishing a communicationslink between the computers may be used.

Referring again to FIG. 1, in illustrative examples, the camera 10 mayinclude a light sensor 50. Light sensor 50 may be used to determineambient light exposure. Ambient light exposure may be useful indetermining whether a flash should be used. Camera 10 may also includean image sensor 70. Image sensor 70 may include any component suitablefor capturing digital image data, such as for example, a charge coupleddevice (CCD), complementary metal oxide semiconductor (CMOS), and thelike. In some examples, the image sensor 70 may use associated imagesensor logic 75 to analyze captured images to develop the image data. Inone example, the camera 10 may also include a secondary image sensor 77.For example, secondary image sensor 77 may have a lower resolution thanimage sensor 70.

In some examples, camera 10 may further include a focus component 60.The focus component 60 may focus an image on the image sensor 70.Additionally, the focus component 60, may determine variousscene-related distances, such as for example the distance to a primarysubject and/or the distance to background objects. In one example, thefocus component 60 may use associate focus component logic 65 todetermine the various scene-related distances.

Still further, the camera 10 may include an auxiliary illumination, orflash component 80, which may generate a photographic flash insubstantial coincidence with the recording of digital image data byimage sensor 70 upon actuation by a user, such as for example,depression of a shutter button. Flash component 80 may be formed intothe camera housing or configured as a removable attachment to the camera10. A flash may be selectively generated by flash component 80 inresponse to the light sensor 50, such as upon the light sensor 50sensing that the ambient light is below a threshold level, or uponmanual input from a user of the camera 10. In some examples, theprocessor 30 may accept scene-related information from one or more ofthe aforementioned sensors/components to be recorded with the digitalrecord for use in pre-processing and/or post-processing of digital imagedata.

In some examples, digital image data and/or data associated therewithmay be stored in a data storage component 90, which may comprise anysuitable computer memory such as, for example, dynamic random accessmemory or a non-volatile memory. A removable storage device 95, such asfor example a CF card, SD card, or the like may further be used to storedigital image data and/or data associated therewith.

In some examples, a flash mitigation system 100 may be implemented inany computing and/or processing environments, including but not limitedto computer software, firmware, device driver, digital electroniccircuitry or computer hardware, or any combination of these. In oneexample, the flash mitigation system 100 may be integral to the camera10 and may comprise a set of instructions running on processor 30. Insome examples, on the basis of pre-stored information and/orscene-related information received by processor 30, flash mitigationsystem 100 may model properties and/or characteristics of shadow effectsresulting from a camera flash and appearing on one or more backgroundobjects. Flash mitigation system 100 may also approximate camera settingadjustments to eliminate or substantially reduce shadow effects incaptured image data. While the present disclosure is described withrespect to examples where shadow effects may be produced as a result ofa camera flash, it is to be appreciated that the systems and methods ofthe present disclosure may be employed to mitigate shadow effectsresulting from any illumination source.

FIG. 3 depicts a process flow diagram of operation of a flash mitigationsystem 100 in accordance with some examples of the present disclosure.Flash mitigation system 100 may acquire scene-related data during apre-capture phase (block 110). Pre-capture phase may refer to a phase,which occurs prior to capturing of image data, in which the cameradetects or senses environment parameters. For example, ambient lightlevels may be detected via light sensor 50 or image sensors 70 or 77. Asa further example, the distance to the primary subject and/or thedistance to one or more background objects may be determined via thefocus component 60 or by measuring reflected light from flash component80 using light sensor 50 or via image sensors 70 or 77. As yet anotherexample, characteristics of shadows resulting from a pre-flash may bedetected via one or more device components, such as for example thefocus component 60, the image sensor 70, and/or the secondary imagesensor 77. The pre-flash may be effected using the flash component 80.Alternatively, the pre-flash may be effected using other means. Usingpre-stored data, such as a flash offset displacement discussed withrespect to FIG. 4 a, and/or the acquired scene-related data, the systemmay then model shadow effects likely to be cast on background objects asa result of auxiliary illumination (block 120). On the basis of themodeled shadow effects, the flash mitigation system 100 may modify oneor more camera settings such that shadows cast upon one or morebackground objects as a result of the flash are eliminated orsubstantially reduced (block 130). After camera setting modification,image data may be captured or recorded by the image sensor 70 and storedto the data storage 90 and/or the removable data storage 95 (block 140).

As an initial matter, in some examples, one or more cameracharacteristics may be pre-stored to the data storage 90. FIG. 4 adepicts a camera 300 suitable for use with systems in accordance withsome examples disclosed herein. As shown, the camera 300 may include abuilt-in flash 310 a and/or a removable flash 310 b which are offsetfrom a lens 320 by a displacement d1a and a displacement d1b,respectively (the “flash offset displacement(s)”). In some examples, thecamera 300 may be configured to sense whether built-in flash 310 a orremovable flash 310 b is employed. As discussed above, the flash offsetdisplacement, at least in part, causes shadow effects resulting from thecamera flash to be visible in a captured image. In some example, flashoffset displacements may be pre-stored to data storage 90.Alternatively, any useful characteristics of camera 10 may be pre-storedto data storage 90.

In some examples, acquisition of data during a pre-capture phase (block110) may include acquisition of one or more parameters which allow formodeling/estimating of shadow effects resulting from the camera flash.In some examples, one or more scene-related distances may be estimatedand stored to the data storage 90.

FIG. 4 b depicts a diagram of an image capture scene in accordance withan illustrative example of the present disclosure. As shown, a user 401may capture an image of a primary target 404, or focal point, using acamera 402 with the aid of auxiliary illumination 403. As also shown,the auxiliary illumination may result in a shadow 405 being cast on anobject 406 in the scene background. In some examples, a focusingcomponent of the camera 402 may approximate the distance to the primarytarget, shown as d2. In further examples, the focusing component mayscan the scene to approximate the distance to one or more backgroundobjects, such as a distance d3 between the camera 402 and a backgroundobject 406. In some examples, either or both of the distance to theprimary target and the distance to background objects may be stored tothe data storage 90 or other suitable storage location. Generally,distances d1, d2, and/or d3 may be used to determine an angle at whichshadow effects will be cast on a background object and/or more reliablypredict the location of shadow effects in captured image data.

In further examples, data acquisition during pre-capture phase (block110) may include acquiring data associated with a pre-flash. Apre-flash, generally, may refer to a flash which is emitted from thecamera prior to and separate from the flash emitted in substantialcoincidence with the recording of captured image data. Generation of apre-flash may create shadow effects likely to appear in captured imagedata. In some examples, upon pre-flash generation, the system mayretrieve data relating to characteristics/properties of producedshadows, such as for example, the darkness, shape, and/or size of theshadow. In illustrative examples, an edge detection algorithm is used todetermine the boundary of the shadow and thus which pixels in theacquired image are impacted by the shadow. In various examples, insubstantial coincidence with the pre-flash, the image sensor 70 and/orthe secondary image sensor 77 may record one or more pre-capture imagesof the scene. The pre-capture images may be stored to the data storage90 for analysis by flash mitigation system 100. It is to be understoodthat the pre-flash of the present disclosure may also be employed aspart of a red-eye reduction system, as is known in the art, oralternatively, may be separate from such systems.

In some examples, using pre-stored data and/or data gathered during thepre-capture phase, the flash mitigation system 100 may model the shadoweffects cast upon background objects (block 120). For example, in oneexample, on the basis of any or all of the flash offset distance,distance to the primary target, distance to one or more backgroundobjects, pre capture images, or combinations thereof, the system maymodel the shadow effects behind the primary target, such as for example,the darkness, size, and/or extent of the shadow. The location of theshadowed pixels within the image may be estimated by using the anglebetween each pixel in the image related to the primary target 404, theillumination source, and the image sensor and then modeling theprojection of the illumination source along this angle to the backgroundobject 406. This angle can be calculated from the displacement d1 andthe distance d2 and then projected to distance d3. Image processingtechniques such as edge detection, can be used to refine the estimationof which pixels are in shadow.

It will be appreciated that a digital image comprises a plurality ofindividual pixels of varying color and shade and that groupings of thesepixels are what make up the digital image. In an illustrative example,modeling of the flash effects (block 120) may include identifying theindividual pixels or region of pixels in one or more stored pre-captureimages which correspond to a shadow area. In this regard, the flashmitigation system 100 may include a pixel locator for locating pixels ina pre-capture image having properties indicative of shadowing. In someexamples, the identified pixels may, at least in part, define a modeledshadow. Alternatively, the flash mitigation system 100 may include anycomponents useful for detecting and eliminating shadow effects.

In some examples, on the basis of the modeled shadow, the flashmitigation system 100 may automatically adjust one or more camerasettings such that the shadow effects are eliminated or substantiallyreduced (block 130). Camera settings can be adjusted on a per pixel,per-line, or per group-of-pixel basis in order to compensate for theillumination differential caused by the flash-induced shadow. Camerasettings that may be adjusted include any conventional camera settingssuch as, for example, pixel gain, pixel offset, shutter speed, exposure,ISO settings, and the like. Alternatively, any appropriate camerasettings may be adjusted. In some examples, one or more of the camerasettings may be pre-set at default values. In further examples, thecamera settings may be modified manually by a user and/or automaticallyby the camera.

The flash mitigation system 100 may adjust the properties of theindividual pixels or region of pixels corresponding to the area of themodeled shadow effects. In this regard, the flash mitigation system 100may include a pixel modifier for modifying the properties of identifiedpixels. In some examples, flash mitigation system 100 may individuallyand dynamically adjust the gain of each identified pixel or group ofpixels, for example, by increasing the gain to compensate for lowcontrast resulting from shadow effects. In one example, flash mitigationsystem 100 may individually and dynamically adjust the exposure of eachidentified pixel or group of pixels, for example by increasing theexposure to compensate for darkness resulting from shadow effects. Thesystems for adjusting gain and exposure described herein are in contrastto known systems, which adjust gain on an image basis (i.e., the gainfor each of the pixels that comprise an image are adjusted by the sameamount) as opposed to an individual pixel or group of pixels basis. Itis to be appreciated that for purposes of post-processing of image datacaptured in accordance with the systems and methods of the presentdisclosure, the individual pixel gain and/or exposure adjustments mayneed to be known. In this regard, as will be discussed in greater detailbelow, metadata comprising, at least in part, individual pixeladjustments may be stored to data storage 90 and associated with thecaptured image data.

In illustrative examples, after camera settings have been modified(block 130), digital image data may be captured by image sensor 70(block 140). The captured image data may be displayed on an imagedisplay, stored to data storage 90, stored to removable storage 95,and/or downloaded to another device, such as a personal computer,server, or printer via an image output component, or otherwiseprocessed.

FIG. 5 illustrates a block diagram of an example computer programproduct 501 arranged in accordance with the present disclosure. In someexamples, as shown in FIG. 5, computer program product 501 includes asignal bearing medium 503 that may also include programming instructions505. Programming instructions 505 may be arranged to pre-store one ormore image capture device characteristics in a data storage. Programminginstructions 505 may also be arranged to acquire data in a pre-capturephase. Further, programming instructions 505 may also be arranged tomodel shadow effects based on either or both of the pre-stored data andthe acquired data. Still further, programming instructions 505 may bearranged to modify one or more image capture device settings based onthe modeled shadow effects. Further yet, programming instructions 505may be arranged to record image data with the image sensor, whereinillumination by a flash is provided substantially coincident withrecording of the image data.

Also depicted in FIG. 5, in some examples, computer product 501 mayinclude one or more of a computer readable medium 506, a recordablemedium 508 and a communications medium 510. The dotted boxes aroundthese elements may depict different types of mediums that may beincluded within, but not limited to, signal bearing medium 503. Thesetypes of mediums may distribute programming instructions 505 to beexecuted by computer devices including processors, logic and/or otherfacility for executing such instructions. Computer readable medium 506and recordable medium 508 may include, but are not limited to, aflexible disk, a hard disk drive (HDD), a Compact Disc (CD), a DigitalVideo Disk (DVD), a digital tape, a computer memory, etc. Communicationsmedium 510 may include, but is not limited to, a digital and/or ananalog communication medium (e.g., a fiber optic cable, a waveguide, awired communication link, a wireless communication link, etc.).

In some particular examples, as shown in the schematic of FIG. 6 a, acomputer system 200 may include a processor 205 configured forperforming an example of a method for minimizing shadow effects within adigital image in accordance with some examples of the presentdisclosure. In other examples, various steps or portions of varioussteps of the method may be performed outside of the processor 205. Invarious examples, the method may include pre-storing one or more imagecapture device characteristics in the data storage 90 (block 210). Themethod may then include acquiring data in a pre-capture phase (block220). Next, the method may include modeling shadow effects based oneither or both of the pre-stored data and the acquired data (block 230).The method may then include modifying one or more image capture devicesettings based on the modeled shadow effects (block 240). Finally, themethod may include recording image data with the image sensor 70,wherein illumination may be provided substantially coincident withrecording of the image data (block 250).

In another particular example, as shown in the schematic of FIG. 6 b, acomputer system 300 may include a processor 305 configured forperforming an example of a method for minimizing shadow effects within adigital image in accordance with some examples of the presentdisclosure. In other examples, various steps or portions of varioussteps of the method may be performed outside of the processor 305. Invarious examples, the method may include acquiring data in a pre-capturephase (block 310). The method may then include modeling shadow effectsbased on the acquired data (block 320). Next, the method may includemodifying one or more image capture device settings based on the modeledshadow effects (block 330). The method may then include accounting forthe modeled shadow effects on a pixel-by-pixel basis (block 340).Finally, the method may include recording image data with the imagesensor, wherein illumination is provided substantially coincident withrecording of the image data (block 350).

In some examples, non-image information, or metadata, may be storedalong with the captured image data. The metadata may be stored as partof non-image header data in a digital image file, or may be stored inany form of digital memory associated with the digital image file.Stored metadata may be used to support post-processing of the digitalimage data, and may include any information relating to the capturedimage data. For example, metadata may include any or all of the offsetdisplacement, distance to primary targets, distance to one or morebackground objects, pre-image data, shadow model data, camera settings(e.g., gain adjustments for each pixel), and the like. Alternatively,metadata may include any information useful for post-processing of thecaptured image data.

FIG. 7 depicts a schematic diagram of an image capture system 400 inaccordance with some examples of the present disclosure. One or morecomponents of the image capture system 400, such as image sensor 410 andfocusing component 420 may be used separately or in combination toacquire data relating to a primary target 430 during a pre-capturephase. In some examples, data acquired by the image sensor 410 maycomprise image data 412 and/or pre-capture data 414. As previouslydescribed, the image sensor 410 may use associated image sensor logic413 to analyze captured images to develop the image data 412 orpre-capture data 414. Data acquired from the focusing component maycomprise pre-capture data 416. As previously described, the focusingcomponent 420 may use associate focusing component logic 417 to analyzecaptured images to develop pre-capture data 416. The acquired data,including any of the pre-capture data 414, the image data 412, or thepre-capture data 416 may be transferred from the image sensor 410 andthe focusing component 420 to a processor 440 via data buses 415 and425, respectively. The processor 440 may model shadow effects based onthe acquired data. The processor 440 may modify one or more imagecapture device settings, such as one or more settings of the imagesensor 410, which may be communicated to the image sensor 410 andassociated logic 413 via data bus 415. The processor 440 may account forthe modeled shadow effects on a pixel-by-pixel basis. The image sensor410 may record image data relating to the primary target 430.Illumination may be generated by a flash component 450 before and/orduring the recording of image data.

The foregoing describes various examples of pre-processing systems andmethods of mitigating defects in captured digital image data resultingfrom the use of an auxiliary illumination source. Following are specificexamples of systems and methods of minimizing shadow effects within adigital image. These are for illustration only and are not intended tobe limiting.

The present disclosure generally relates to systems for capturingimages. The described systems may include an image sensor, a flash forproviding illumination, a data storage, and a processor operativelyassociated with the data storage. The processor may be operativelyassociated with the data storage and adapted to execute computerimplemented instructions to pre-store one or more image capture devicecharacteristics in the data storage, acquire data in a pre-capturephase, model shadow effects based on either or both of the pre-storeddata and the acquired data, modify one or more image capture devicesettings based on the modeled shadow effects, and record image data withthe image sensor during illumination by the flash.

In some further described systems, an image capture system may includean image sensor, a flash for providing illumination, a data storage, anda processor operatively associated with the data storage. The processormay be operatively associated with the data storage and adapted toexecute computer implemented instructions to acquire data in apre-capture phase, model shadow effects based on the acquired data,modify one or more image capture device settings based on the modeledshadow effects, account for the modified shadow effects on apixel-by-pixel basis, and record image data with the image sensor.Illumination may be provided substantially coincident with recording ofthe image data.

The present disclosure also generally relates to computer accessiblemediums having stored thereon computer executable instructions forprocessing an image when the executable instruction are executed by aprocessing unit of an image capture device that includes an image sensorand a data storage. The instructions may cause the processing unit toperform a process of reducing shadow effects within a digital image. Insome examples, the described processes may include pre-storing one ormore image capture device characteristics in the data storage, acquiringdata in a pre-capture phase, modeling shadow effects based on either orboth of the pre-stored data and the acquired data, modifying one or moreimage capture device settings based on the modeled shadow effects, andrecording image data with the image sensor. The illumination may beprovided substantially coincident with recording of the image data.

The present disclosure further generally relates to methods forminimizing shadow effects within a digital image. In some examples, themethods may include acquiring data in a pre-capture phase, modelingshadow effects on the basis of the acquired data, modifying one or moreimage capture device settings on the basis of the modeled shadoweffects, accounting for the modeled shadow effects on a pixel-by-pixelbasis, and recording image data with an image sensor during illuminationby the flash.

The present disclosure is not to be limited in terms of the particularexamples described in this application, which are intended asillustrations of various aspects. Many modifications and variations maybe made without departing from its spirit and scope, as will be apparentto those skilled in the art. Functionally equivalent methods andapparatuses within the scope of the disclosure, in addition to thoseenumerated herein, will be apparent to those skilled in the art from theforegoing descriptions. Such modifications and variations are intendedto fall within the scope of the appended claims. The present disclosureis to be limited only by the terms of the appended claims, along withthe full scope of equivalents to which such claims are entitled. It isto be understood that this disclosure is not limited to particularmethods, reagents, compounds compositions or biological systems, whichcan, of course, vary. It is also to be understood that the terminologyused herein is for the purpose of describing particular examples only,and is not intended to be limiting.

With respect to the use of substantially any plural and/or singularterms herein, those having skill in the art may translate from theplural to the singular and/or from the singular to the plural as isappropriate to the context and/or application. The varioussingular/plural permutations may be expressly set forth herein for sakeof clarity.

It will be understood by those within the art that, in general, termsused herein, and especially in the appended claims (e.g., bodies of theappended claims) are generally intended as “open” terms (e.g., the term“including” should be interpreted as “including but not limited to,” theterm “having” should be interpreted as “having at least,” the term“includes” should be interpreted as “includes but is not limited to,”etc.). It will be further understood by those within the art that if aspecific number of an introduced claim recitation is intended, such anintent will be explicitly recited in the claim, and in the absence ofsuch recitation no such intent is present. For example, as an aid tounderstanding, the following appended claims may contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimrecitations. However, the use of such phrases should not be construed toimply that the introduction of a claim recitation by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim recitation to examples containing only one suchrecitation, even when the same claim includes the introductory phrases“one or more” or “at least one” and indefinite articles such as “a” or“an” (e.g., “a” and/or “an” should be interpreted to mean “at least one”or “one or more”); the same holds true for the use of definite articlesused to introduce claim recitations. In addition, even if a specificnumber of an introduced claim recitation is explicitly recited, thoseskilled in the art will recognize that such recitation should beinterpreted to mean at least the recited number (e.g., the barerecitation of “two recitations,” without other modifiers, means at leasttwo recitations, or two or more recitations). Furthermore, in thoseinstances where a convention analogous to “at least one of A, B, and C,etc.” is used, in general such a construction is intended in the senseone having skill in the art would understand the convention (e.g., “asystem having at least one of A, B, and C” would include but not belimited to systems that have A alone, B alone, C alone, A and Btogether, A and C together, B and C together, and/or A, B, and Ctogether, etc.). In those instances where a convention analogous to “atleast one of A, B, or C, etc.” is used, in general such a constructionis intended in the sense one having skill in the art would understandthe convention (e.g., “a system having at least one of A, B, or C” wouldinclude but not be limited to systems that have A alone, B alone, Calone, A and B together, A and C together, B and C together, and/or A,B, and C together, etc.). It will be further understood by those withinthe art that virtually any disjunctive word and/or phrase presenting twoor more alternative terms, whether in the description, claims, ordrawings, should be understood to contemplate the possibilities ofincluding one of the terms, either of the terms, or both terms. Forexample, the phrase “A or B” will be understood to include thepossibilities of “A” or “B” or “A and B.”

In addition, where features or aspects of the disclosure are describedin terms of Markush groups, those skilled in the art will recognize thatthe disclosure is also thereby described in terms of any individualmember or subgroup of members of the Markush group.

As will be understood by one skilled in the art, for any and allpurposes, such as in terms of providing a written description, allranges disclosed herein also encompass any and all possible subrangesand combinations of subranges thereof. Any listed range may be easilyrecognized as sufficiently describing and enabling the same range beingbroken down into at least equal halves, thirds, quarters, fifths,tenths, etc. As a non-limiting example, each range discussed herein maybe readily broken down into a lower third, middle third and upper third,etc. As will also be understood by one skilled in the art all languagesuch as “up to,” “at least,” “greater than,” “less than,” and the likeinclude the number recited and refer to ranges which may be subsequentlybroken down into subranges as discussed above. Finally, as will beunderstood by one skilled in the art, a range includes each individualmember. Thus, for example, a group having 1-3 cells refers to groupshaving 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers togroups having 1, 2, 3, 4, or 5 cells, and so forth.

While various aspects and examples have been disclosed herein, otheraspects and examples will be apparent to those skilled in the art. Thevarious aspects and examples disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

What is claimed is:
 1. An apparatus, comprising: a processing unitconfigured to perform or cause to be performed: during a pre-capturephase: acquire scene-related data of a scene via an image sensor,wherein acquisition of the scene-related data includes recordation of apre-capture image at a memory by use of a pre-flash of a flash; modelshadow effects cast upon a background object based on the acquiredscene-related data; and configure one or more device settings to altershadow effects based on the modeled shadow effects; and after thepre-capture phase, record, at the memory, image data of the scenereceived via the image sensor and illuminated by the flash using the oneor more image capture device settings.
 2. The apparatus of claim 1,wherein the processing unit is further configured to perform or cause tobe performed: during the pre-capture phase, initialize the one or moredevice settings.
 3. The apparatus of claim 1, further comprising theimage sensor, which is a first image sensor, and a second image sensor,wherein the processing unit is further configured to perform or cause tobe performed: during the pre-capture phase, acquire additionalscene-related data via the first image sensor and the second imagesensor; model shadow effects cast upon the background object based onthe additional scene-related data; and after the pre-capture phase,record, at the memory, additional image data received via the secondimage sensor.
 4. The apparatus of claim 1, wherein the processing unitis configured to perform or cause to be performed: acquire thescene-related data at least in part by acquisition of a distance to aprimary target of the scene as determined by the image sensor.
 5. Theapparatus of claim 4, wherein the processing unit is configured toperform or cause to be performed: acquire the scene-related data atleast in part by acquisition of a distance to the background object ofthe scene as determined by the image sensor.
 6. The apparatus of claim1, wherein the processing unit is configured to perform or cause to beperformed: model shadow effects cast upon the background object at leastin part by estimation of a location of shadowed pixels on the backgroundobject based on the acquired scene-related data.
 7. The image capturesystem of claim 1, wherein the processing unit is configured to performor cause to be performed: model shadow effects cast upon the backgroundobject at least in part by approximation of one or more of darkness,size, or shape of an identified shadow on a pixel of the backgroundobject.
 8. The apparatus of claim 6, wherein the processing unit isconfigured to perform or cause to be performed: estimate a location ofshadowed pixels on the background object at least in part by estimationof an angle between a pixel of a primary target of the scene, the flash,and the image sensor.
 9. The apparatus of claim 8, wherein theprocessing unit is configured to perform or cause to be performed:estimate a location of shadowed pixels on the background object at leastin part by a model of a projection of the flash along the angle to thebackground object.
 10. The apparatus of claim 8, wherein the processingunit is configured to perform or cause to be performed: estimate anangle between a pixel of a primary target of the scene, the flash, andthe image sensor based at least in part on a distance to the backgroundobject, a distance between the flash and the image sensor, or both. 11.The apparatus of claim 1, wherein the processing unit is configured toperform or cause to be performed: during the pre-capture phase, acquirescene-related data of the scene via the image sensor at least in part bystorage, at the memory, of one or more pre-capture images recorded bythe image sensor contemporaneous with the pre-flash of the flash.
 12. Anon-transitory computer accessible medium having stored thereon computerexecutable instructions that, in response to execution by one or moreprocessing units, cause the one or more processing units to perform orcontrol performance of: during a pre-capture phase: operate an imagesensor to obtain scene related data in the pre-capture phase including apre-capture image of a scene by use of a pre-flash of a flash; modelshadow effects cast upon a background object based on the scene-relateddata; and configure an image capture device setting based on the modeledshadow effects; and following the pre-capture phase, record image dataof the scene illuminated by the flash via the image sensor using theconfigured image capture device setting.
 13. The computer accessiblemedium of claim 12, wherein the executable instructions, which inresponse to execution by the one or more processing units, further causethe one or more processing units to perform or control performance of:store individual pixel adjustment data associated with the scene basedon the modeled shadow effects.
 14. The computer accessible medium ofclaim 13, wherein the executable instructions, which in response toexecution by the one or more processing units, further cause the one ormore processing units to perform or control performance of: adjustindividual pixels of the image data based on the stored individual pixeladjustment data.
 15. The computer accessible medium of claim 12, whereinthe executable instructions, which in response to execution by the oneor more processing units, cause the one or more processing units toperform or control performance of: estimate a location of shadowedpixels on the background object based on the scene-related data.
 16. Amethod, comprising: during a pre-capture phase: acquiring scene-relateddata including a pre-capture image received via an image sensor with apre-flash of a flash; modeling shadow effects cast upon at least onebackground object based on the acquired scene-related data; andmodifying an image capture setting based on the modeled shadow effects;and recording image data of the scene with the image sensor using themodified image capture setting during illumination of the scene by theflash.
 17. The method of claim 16, further comprising initializing theimage capture setting during the pre-capture phase.
 18. The method ofclaim 16, wherein the image sensor is a first image sensor, the methodfurther comprising: during the pre-capture phase, acquiring additionalscene-related data via a second image sensor, wherein modeling shadoweffects cast upon the at least one background object is further based onthe additional scene-related data; and after the pre-capture phase,recording additional image data received via the second image sensor.19. The method of claim 16, wherein modeling shadow effects cast uponthe at least one background object based on the acquired scene-relateddata includes estimating a location of shadowed pixels on backgroundobjects based on the acquired scene-related data.
 20. The method ofclaim 19, wherein estimating the location of the shadowed pixels on thebackground objects based on the scene-related data includes: estimatingan angle between at least two of a pixel of a primary target of thescene, the flash, or the image sensor; and modeling a projection of theflash along the angle to the background objects.